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User satisfaction is one of the system use variables which is affected by the 
system creation variables in the information system (IS) success 
measurements, especially in the mandatory use of IS. This paper reports the 
relational variable assessments between three variables of the system creation 
dimension towards the user satisfaction variable in the library information 
system (LIS) implementation of a sampled university in Indonesia. 
Practically, the measurement has never been done since the early system 
implementation in the institution. The study focused on the status of the user 
satisfaction construct and what are the factors which influenced the construct. 
The used measurement model was adopted and adapted from the DeLone and 
McLean’s IS success model. A total of 185 respondents were selected in this 
study using multi-stage purposeful random sampling. The researchers used 
the partial least squares structural equation modeling (PLS-SEM) with the 
SmartPLS version 2.0 for analyzing the collected data. Findings of the study 
showed that users of the LIS were sufficiently satisfied and the proposed 
hypotheses were accepted. In terms of the adopted model, besides the 
findings theoretically proved that the user satisfaction construct has affected 
by the system creation constructs; the findings may also have proposed the 
practical recommendations to the sampled institution for the next LIS 
improvements in particular. 
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1. INTRODUCTION 


It is like the other IS implementations in the higher education institution’s (HEI) business functions, 
the implementation of an LIS is also indispensable now. It is related to the essential functions of the system 
for the educational institutions, especially among the developing countries, like Indonesia. It is for supporting 
the research and data management services [1]. LIS is useful and helpful for students and lectures in the 
learning and research works. One of the issues of the IS success measurement is the user satisfaction [2-8]. 
Despite the fact that the previous studies [9, 10] indicated that the user satisfaction of LIS is significantly 
affected by factors of the system creation dimension and the satisfaction factors also affects the system 
success, in terms of the IS success model [2-8]; the phenomenon seems to be ignored in many LIS 
implementations among HEIs in Indonesia, including in the sampled institution. Despite LIS has been 
implemented for almost ten years, the preliminary study findings indicated that the system is not yet to be 
done by the stakeholders. Therefore, the user satisfaction assessment is still indispensable to be performed 


here. 
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This study was performed to know the status of the user satisfaction and to assess what factors that 
influenced the user satisfaction variable of the LIS. The objectives were to present status based on the user 
perspectives and to examine the influential factors referring to the used research model. In this study, the 
researchers adopted four constructs of the DeLone and McLean’s [2] IS success model. 

The paper is organized into five sections. In the introduction section, the authors explain the study 
background. In the literature review section, the scholars explain the literature review results and elucidate 
the theoretical framework of the used research model. In the research method section, the methodological 
explanations of the study are described sequentially following the research procedure. In results and analysis 
section, the researchers present the experimental results, its analysis, and the interpretation. Finally, the 
article is closed with the conclusion section. 


2. LITERATURE REVIEW 

The DeLone and McLean’s [2] IS success model which is developed by the authors based on the 
process and the causal relationship of the dimensions in the model [11, 12]. Many studies have been referring 
and using the model [11, 12]. The popularity of the models shows strong evidence of the need to integrate 
research findings in a comprehensive manner in the IS success measurement topic. This model becomes a 
concern the researchers, as like Seddon [13] who criticize the model. According to Seddon, the main problem 
of the IS success model is trying to combine the processes and causal explanation of the success of IS in their 
models. Thus their model mixes between a process model and the variance model. Another criticism is that 
the system use is a behavior, so it should be removed as a success measurement of the causal model. 

However, the authors of the IS success mode argue that system use must precede the impacts and 
benefits. They believe that the use of a system is an appropriate measurement to measure success in most 
cases [11, 12], as it is presented by many previous studies, especially in the 10 years since the model was first 
introduced. By examining more than 100 articles published in well-known journals of IS, e.g., the Journal of 
Management Information Systems, and MIS Quarterly since 1993, DeLone and McLean [11, 12] revise their 
model which now consists of six variables: information quality, system and service quality, intention to use / 
use, user satisfaction, and net benefits. Referring to the study of DeLone and McLean’s [2] IS success model 
theory, the researchers adopted the 4 variables of the model, especially variables associated with user 
satisfaction, i.e., information quality (INQ), system quality (SYQ), service quality (SVQ), and user 
satisfaction (USF) (Figure 1). In terms of the mandatory issues [13-15], the usability variable was not used 
here. In detail, Table 1 shows the list of the used indicators of the model and Figure 1 shows the three 
hypothetical paths between three independent variables with the dependent one. 























Table 1. References of the indicators 
Code Indicators References 
INQI Accuracy 
INQ2 Consistency DeLone & McLean [2] i 
INQ3 Reliability Urbach et al. [16] Se 
INQ4 Timeliness Al-Debei et al. [17] H2 
INQ5 Customisation 
sYQI Ease of bei DeLone & McLean [2] ae 
SYQ2 Reliability H 
iver Urbach et al. [16] 3 
SYQ3 Flexibility 
ae Kwan [18] 
SYQ4 Searchability 
: Yang et al. [19] 
oe Sony Al-Debei et al. [17] 
SYQ6 Accessibility : 
SVQ1 Reliability DeLone & McLean [2] : 
SVQ2 Responsiveness Roses et al. [20] Figure 1. The research model 
SVQ3 Empathy Gorla et al. [21] 
SVQ4 Assurance Al-Debei et al. [11, 17] 
Leet Acleaquacy DeLone & McLean [2] 
USF2 Effectiveness Petter-et al, [11] 
USF3 Efficiency : 


USF4 — Overall satisfaction Sublyaltos: stan] 





3. RESEARCH METHOD 

This study was performed across its eight gradual stages, from the literature study in the first step 
into the report writing in the last one. Figure 2 presents the sequential implementation of the procedure, 
including the output of each stage. The population of the study (N= + 21.067) was identified based on the 
human resource department database of the year 2016 in the sampled institution. The survey was conducted 
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by using a multi-stage purposeful random sampling technique [22-23] . Around 185 valid answers were then 
collected by online invitation survey. The first step was purposive sampling with the criteria that the 
respondents are the LIS users and the second step was simple random sampling technique. The instrument of 
the data collection technique was an online questionnaire using the Google Doc. This data collection 
instrument consisted of two main parts, i.e. the invitation letter page and question pages. The question part 
comprised of the six demographic questions and its 19 five-point Linkert scale questionnaires. The collected 
data were then processed by using the Microsoft Excel 2007 and the IBM SPSS version 20 to prepare for the 
analysis stage. 


Analysis 


(1) (2) 
Literature Model 


Review | |Development 





Research 
Program 





Figure 2. The research procedure 


Sequentially, the researchers then analyzed the processed data by using the PLS-SEM method with 
the SmartPLS 2.0 statistic software [16, 24-29]. In the descriptive part of the analysis stage, the researchers 
analyzed the demographic data by using the Microsoft Excel 2007 in order to look the dissemination of the 
collected data. In addition, the inferential analysis was then performed through two assessments; i.e., the 
measurement and structural model assessments following the above mentioned the PLS-SEM studies. The 
first assessment was carried out by employing the indicator reliability, internal consistency reliability, 
convergent validity, and the discriminant validity examinations, in order to assess the psychometric 
properties of the outer model. On the other side, the second one was performed by implementing the path 
coefficient (f), the coefficient of determination (R2), t-test, effect size (f2), predictive relevance (Q2), and the 
relative impact (q2) assessments for evaluating its inner model. Furthermore, the interpretation stage was 
then focused on the hypothetical assessment among the six above mentioned assessments, referring to the 
determined research programs. Moreover, besides the results of the descriptive analysis, the previous findings 
of the similar studies were also discussed in the reporting stage. 


4. RESULTS AND ANALYSIS 
4.1. The demography information 

Table 2 shows the demographic information, including gender, faculty, semester, and respondent 
opinion about library information system. The information indicated that most of them (73.5%) are the 
female respondents and the rest ones were male, majority of the people (44.3%) are from the science and 
technology students, the distribution is dominated by the final year students above 8th semester (33%), and 
then most of them (66.5%) use the systems only less than 3 times a week. Furthermore, majority respondents 
(92.5%) felt helped by the system, 22.2% of them even said that the system is very helpful in completing the 
work/study and 45.3% of the people were quite satisfied in using the system. In regard to the first purpose 
and objective of the study, it can be seen that the above-mentioned results indicated the user satisfaction 
status of the LIS implementation in the sampled institutions. 


4.2. Results of the Statistical Analysis 

First, results of the measurement model assessment show statistically that the measurement model 
indicates the good psychometric properties, even with two indicator rejections (i.e, SYQ3 and SYQ6). In 
detail, Table 3, Table 4 and Figure 3 describe the results. 

a) Individual item reliability was evaluated by checking the value of standardized loading factor. The value 
describes the correlation between each measurement (indicators) item and its construct. The loading 
factor value above 0.7 considered as ideal. After three assessment times, SYQ3 and SYQ6 were then 
rejected because their loadings are under the threshold value (Table 3 and Figure 1). 
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Figure 3. Results of the measurement model assessment 























Table 2. Demographic information Table 3. Results of the outer model assessments 
Measures Items % Cross Loadings 
Sere Female 735 Items “No. svq. syq usr “YE SR 
Male 26.5 INQ! 0.739-—s:0.393 0.456: 0.422 
Islamic Studies 0.5 INQ2 0.804 —-0.462,-—s-:0.474~—s (0.473 
Education 13 INQ3 0.788 )=—s (0.454. (0.491 (0.505. 0.574 (0.87 
Law 9.7 INQ4 0.641 0.416 0.395 0.37 
Humaniora 3.2 INQ5 0.804 =—:0.618 »=—-0.653 0.631 
Philosophy 2.7 SVQI 0461 0.781 0.552 0.59 
Faculty Communication 2.1 SVQ2 0.601 90.851 0.659 = 0.587 0.667 0.889 
Science and Technology 44.3 SVQ3 0.416 «80.825 0.512 O51 . : 
Economy and Business 9.7 SVQ4 0.565 0.81 0.575 0.628 
Social and Political Science 3.8 SYQI 0.44 0.511 0.799 0.556 
Psychology 4.3 SYQ2 0.612 0.644 0.835 0.665 0.659 0.885 
Medicine 5.9 SYQ4 0.633 0.589 0.854 (0.686 . . 
1-2 1.6 SYQ5 0.452 0.541 0.756 0.54 
geniccigs 3-5 36.2 USFI 0.558 0.595 0.644 0.855 
6-8 29.2 USF2. 0.59 =—-0.671_—s(0.714 0.918 0.784 0.936 
>8 33 USF3 0.571 ~—-0.623— «0.705 ~—s: 0.901 : . 
; ; <3 66.5 USF4 0.583 (0.637. «0.622 _~—(0.867 
The intensity 3-5 22.2 
of the system 5-10 16 
use in a week > 10 38 
Very helpful 22.2 Tabel 4. Discriminant validity 
Thetoleot Helpful 31.4 Items INQ SVQ SYQ USF 
Te 9 Quite helpful 38.9 INQ 0.758 
Less helpful 7 SVQ 0.631 0.817 
Not helpful 0.5 SYQ 0.667 0.707 0.812 
Very satisfied 6.5 USF 0.65 0.713 0.759 0.886 
User Satisfied 31.4 
satisfaction Quite satisfied 47 
level of LIS Less satisfied 14.6 
Not satisfied 0.5 





b) Internal consistency reliability was examined using composite reliability (CR) with the threshold value 
of 0.7 and above. Table 3 shows that overall CR values of the three variables were above 0.7. 

c) Average variance extracted (AVE) was assessed with the limit value of 0.5 or above. The result can be 
seen in Table 3 which shows AVE values of the three variables were above 0.5. 

d) Discriminant validity was tested by analyzing the cross-loading and comparing it to the root of AVE. 
The size of the cross loading is compared the correlation of indicator with its own construct and other 
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blocks construct. When the correlation between the indicators with its construct is higher than the 
correlation with another block, it indicates that the construct predicts the size of the block is better than 
the other block. The size of other discriminant validity is that the root value of AVE should be higher 
than the correlation between a construct and another construct. The result can be seen in Table 3 and 
Table 4 which shows that the value of cross loading indicator with a construct of the entire variable is 
higher than the correlation to construct another block. Similarly, AVE root value is higher than the 
correlation between a construct with the other construct. 

Second, referring to the descriptions of the previous studies [16, 24-28], the psychometric properties 

of the outer model was then used as the standing point for continuing the inner model assessments. Table 5 

shows that the structure model assessment results by using path modeling, bootstrapping, and blindfolding 

procedures. 

a) Path coefficient (6) was evaluated with threshold value 0.1 or above to identify the significance of the 
path influence in the model. Results of this evaluation indicated that the three links were the significant 
paths. 

b) The coefficient of determination (R2) was evaluated with three thresholds, i.e, around 0.67, about 0.33, 
approximately 0.19 and lower as substantial, average, and weak. Figure 1 presents that the three 
independent variables of the model explained substantially (65.4%) the USF variance. 

c) the t-test was assessed by using bootstrapping method with the two-tailed test with 5% significance level 
to test the hypotheses of the study. Those hypotheses will be accepted if the value has a greater than 
1.96. Table 5 shows that all hypotheses are accepted. 

d) Effect size (f2) was assessed to predict the influence of each variable toward another in the model 
structure with threshold value about 0.02 for a small, 0.15 for medium, or 0.35 large influences. Table 5 
represents that only one path that has a medium effect and the others has small effects. 

e) Predictive relevance (Q?) was evaluated by using blindfolding method to represent predictive relevance 
of the target endogenous variable with threshold value above the zero. Table 5 shows that of all the 
variables are predictive relevance. 

f) Relative impact (q?) was tested by using blindfolding method for measuring the relative influence of a 
certain predictive variables relationship with another variable with a value threshold of about 0.02 for a 
small influence, 0.15 for a medium/moderate influence, and 0.35 for a large influence. Table 5 shows the 
three paths have small influences. 


Table 5. Results of the structural model assessment 
R? f on q Analysis 
R’-in  R*-ex yf Q’-in Q’-ex Yq B t-test R’ f q 
H! 0.173 2.101 0.654 0.639 0.043 0.509 0.498 0.022 Sign Acc Mid. Small Small 
H2 0.299 3.615 0.654 0.576 0.225 0.509 0.449 0.122 Sign Acc Mid. Mid. Small 
H3 0.433 4483 0.654 0.613 0.118 0.509 0.478 0.063 Sign Acc Mid. Small Small 








Hs B t-test 








It can be clearly seen that both demographic and hypothetical results above-mentioned are 
interrelated among others. Results of the structural model measurements proofed the user satisfaction 
indications of the LIS implementation in the sampled institution. Besides, majority respondents (92.5%) felt 
be helped by the system, 22.2% of them even said that the system is very helpful in completing the 
work/study, and 45.3% of the people were quite satisfied in using system; results of the statistical analysis 
also presented that the user satisfaction factor is presented substantially (65.4%) by the information quality, 
system quality, and service quality factors. As it was also presented by several prior IS success measurement 
studies [3-8], [30]. 


5. CONCLUSION 

The success of LIS implementation is indispensable for HEIs in developing countries like Indonesia. 
It is related to the essential functions of the system for the institutions related to support their research and 
data management services. In this case study, the findings indicated that majority of the users are the senior 
students who revealed that LIS is quite helpful and satisfactory for their research works. Despite the fact that, 
results of the measurement model analysis show two indicator rejections (SY3 and SY6), but the outer model 
of the used model had a good psychometric property. Therefore, it was then used as starting point for the 
inner model assessments. In addition, although the three independent variables had the small relative impact 
towards the dependent variable, their path coefficients were significant with the middle level in the 
coefficient of determination. Thus, acceptance of the hypotheses was reliable. Furthermore, it is unsurprising 
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that the outer part of the used model has the psychometric properties and the inner part also presented the 
positive indications. It may in regard to utilization of the popular IS model. In respect of the two indicator 
rejection, it may relate to the used instrument and data in the study. This may be one of the consideration 
points for the similar studies in the future. This case study may not contribute theoretically to the user 
satisfaction measurement topics, but findings of the study may still interesting, in terms of the practical issue 
in the sampled institution. As it is presented by the findings, the three factors have only explained 65.4% of 
the user satisfaction explanation. Therefore, the findings can be one of the next consideration points for 
institutions for further system development, in respect of the user satisfaction points. 
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